Langchain instrumentor implementation #3662
Conversation
copied over __init__.py copied over callback handlers copied over span_attributes file
|
|
|
@haneric00 I think you missed we have already the skeleton for a langchain implementation. Please get in touch with @wrisa and the other genai people so we don't duplicate effort there. |
Thanks Riccardo - Agree with avoiding duplicate efforts. I've left a message in the #otel-genai-instrumentation channel to sync with the relevant folks. |
|
@haneric00 did you get alignment? Is this ready for review? |
Hi, we will be meeting to discuss alignment today at 2pm PST. |
@aabmass so we are aligned, we will be focusing on agentic workflow spans and they will be focusing on LLM invocation spans. For the time being, we will wait for their open PRs to get merged so I will move mine to draft mode for now, and then rebase after their work is merged. |
| if run_id in self.span_mapping: | ||
| span = self.span_mapping[run_id].span | ||
|
|
||
| _set_span_attribute(span, "gen_ai.agent.tool.input", tool_input) |
There was a problem hiding this comment.
Are these attributes in semantic conventions already? do not see them here https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-agent-spans.md
There was a problem hiding this comment.
sorry, these are placeholders until an alternative to the deprecated attribute is ready
| self._handle_error(error, run_id, parent_run_id, **kwargs) | ||
|
|
||
|
|
||
| def on_agent_action(self, |
There was a problem hiding this comment.
I see the example is using langchain agent which is not recommended for production application, please see https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/react/agent.py#L33.
Have you tried langraph agents? on_agent_action and on_agent_finish callbacks are never called when langraph agents are used. Do we know what is to be done there?
There was a problem hiding this comment.
We wanted to keep these callbacks available to cover a wider spread of use cases. We have been testing on both langgraph and langchain applications. I am a little confused on what the issue is, are these callback handlers to be removed?
Description
OpenTelemetry has published an initial verson AI Agent semantic conventions that cover the fundamental attributes for AI Agent spans. However, existing third-party instrumentation solutions (OpenInference, OpenLLMetry, OpenLIT) have significant gaps with official OpenTelemetry GenAI semantic conventions:
Proposed Solution
Build an OpenTelemetry instrumentation library for LangChain that:
This PR encompasses a langchain instrumentation solution that is currently tailored to work with AWS products like Cloudwatch and Bedrock. I want to get this solution onto upstream first and continuously update it to work universally overtime.
Type of change
How Has This Been Tested?
1. Test file 1: test instrumentation works on an agentic application of langchain
2. Test file 2: test instrumentation works on normal chain usage of langchain
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.